Identifying heterogeneous treatment effects for online single-session interventions for adolescent depression: a secondary analysis

Note: Press the button on the upper right of this file to show all codes.

Report: Download 2024_Comps_Part_III_Report.pdf. Git repo can be found here

##  package     * version date (UTC) lib source
##  bookdown      0.37    2023-12-01 [1] CRAN (R 4.3.0)
##  bslib         0.6.1   2023-11-28 [1] CRAN (R 4.3.0)
##  cachem        1.0.8   2023-05-01 [1] CRAN (R 4.3.0)
##  cli           3.6.2   2023-12-11 [1] CRAN (R 4.3.0)
##  digest        0.6.34  2024-01-11 [1] CRAN (R 4.3.0)
##  evaluate      0.23    2023-11-01 [1] CRAN (R 4.3.0)
##  fastmap       1.1.1   2023-02-24 [1] CRAN (R 4.3.0)
##  htmltools     0.5.7   2023-11-03 [1] CRAN (R 4.3.0)
##  jquerylib     0.1.4   2021-04-26 [1] CRAN (R 4.3.0)
##  jsonlite      1.8.8   2023-12-04 [1] CRAN (R 4.3.0)
##  knitr         1.45    2023-10-30 [1] CRAN (R 4.3.0)
##  lifecycle     1.0.4   2023-11-07 [1] CRAN (R 4.3.0)
##  mime          0.12    2021-09-28 [1] CRAN (R 4.3.0)
##  R6            2.5.1   2021-08-19 [1] CRAN (R 4.3.0)
##  rlang         1.1.3   2024-01-10 [1] CRAN (R 4.3.0)
##  rmarkdown     2.27    2024-05-17 [1] CRAN (R 4.3.3)
##  rmdformats    1.0.4   2022-05-17 [1] CRAN (R 4.3.1)
##  rstudioapi    0.15.0  2023-07-07 [1] CRAN (R 4.3.0)
##  sass          0.4.8   2023-12-06 [1] CRAN (R 4.3.0)
##  sessioninfo   1.2.2   2021-12-06 [1] CRAN (R 4.3.0)
##  xfun          0.41    2023-11-01 [1] CRAN (R 4.3.0)
##  yaml          2.3.8   2023-12-11 [1] CRAN (R 4.3.0)
## 
##  [1] /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library

1 Data preparation

First, load dataset and clean data as necessary.

1.1 Collapse categories

Re-code sexual orientation, race, family challenge variables by collapsing some categories with limited number of samples, and generating composite categories.

1.1.0.1 Recode sexual orientation

1.1.0.2 Recode family challenges

1.1.0.3 Recode race

1.2 Latent Class Analysis(LCA)

Apply LCA to find the latent class variables for gender identity and coping strategies, respectively.

1.2.1 Gender identity

1.2.1.1 Step 1: LCA model selection

Compare LCA models for different number of classes.

the LCA model with 3 latent classes has comparable \(G^2\), AIC, BIC. (could also use scree plot). therefore, a model with 3 latent class is selected for gender identity.

Table 1.1: Model Comparison for Different Number of Classes (Gender Identity variable)
Model G2 AIC BIC
2 1348.0556 7529.195 7683.297
3 924.7536 7135.893 7369.704
4 821.6347 7062.774 7376.293
5 616.5568 6887.696 7280.923

1.2.1.2 Step 2: classification error: averaged posterior probability(APP)

print APP, results suggest low classification error(APP>0.7).

## [1] 0.9648380 0.9863283 0.9650341

1.2.1.3 Step 3: Final latent class probabilities

The probability of belonging to a specific class is calculated for each individual, and each individual is classified to a specific class based on the max. posterior probability. Below is the results of the posterior probabilities for all subjects (prob. of being in class \(j\)).

1.2.1.4 Step 4: Assign label to each latent class (gener identity)

Visualize the probabilities of answering yes of each item by latent class (Pr(individual answers yes to an item \(|\) in class \(j\))) to understand the underlying pattern.

1.2.2 Coping strategies

1.2.2.1 Step 1: LCA model selection

Build and compare LCA models

Table 1.2: Model Comparison for Different Number of Classes (Coping strategy variable)
Model G2 AIC BIC
2 29.066630 4548.577 4596.402
3 6.162343 4535.673 4610.067
4 6.058262 4545.569 4646.533
5 1.878940 4551.389 4678.923

1.2.2.2 Step 2: Classification error: averaged posterior probability(APP)

print averaged posterior probability(APP) for coping strategy, results suggest low classification error(APP>0.7).

## [1] 0.8685438 0.7917650 0.9089532

1.2.2.3 Step 3: Final latent class probabilities

Similarly, get the predicted posterior probability of the latent variable for coping strategy.

1.2.2.4 Step 4: Assign label to each latent class (coping strategies)

1.3 Missing values

98.6% of the subjects have complete information.

Tabulate the number and percentage of missing. The missing rate is low, so complete case analysis will be used later.

Check if there is any missing pattern among missing variables: No pattern presents!

1.4 The final working dataset

After some investigation, I decided to do a complete case analysis(CCA), the final working dataset is then generated.

dat_cc <- cope_subset3 %>% 
   filter(recode_race!="Prefer not to answer") %>%  ## 1468
  filter(b_dem_sex!="Prefer not to say" & b_dem_sex!="Other") %>% ## 1447
  filter(orientation!="I do not want to respond") ## 1441

2 Descriptive table (table 1)

A Descriptive table is generated using the 1441 subjects with complete data. Summary stats is stratified by treatment condition.

Demographics Treatment Received
Placebo Control
N = 488
1
Project ABC
N = 489
1
Project Personality
N = 464
1
Baseline CDI mean score(0-2) 1.16 (0.35) 1.15 (0.34) 1.17 (0.36)
Race


    Asian Including Asian Desi 50 (10%) 58 (12%) 50 (11%)
    Black/African-American 33 (6.8%) 40 (8.2%) 36 (7.8%)
    Hispanic/Latinx 57 (12%) 61 (12%) 53 (11%)
    Mixed 74 (15%) 68 (14%) 63 (14%)
    White 274 (56%) 262 (54%) 262 (56%)
Age (yrs)


    13 28 (5.7%) 32 (6.5%) 28 (6.0%)
    14 77 (16%) 81 (17%) 63 (14%)
    15 150 (31%) 156 (32%) 162 (35%)
    16 233 (48%) 220 (45%) 211 (45%)
Biological sex


    Female 434 (89%) 437 (89%) 418 (90%)
    Male 54 (11%) 52 (11%) 46 (9.9%)
Sexual orientation


    Heterosexual 108 (22%) 97 (20%) 106 (23%)
    LGBTQ 309 (63%) 327 (67%) 291 (63%)
    Other 71 (15%) 65 (13%) 67 (14%)
Language


    English 476 (98%) 476 (97%) 450 (97%)
    Other 12 (2.5%) 13 (2.7%) 14 (3.0%)
Gender identity


    Non-binary 102 (21%) 98 (20%) 86 (19%)
    Women/girls 313 (64%) 325 (66%) 306 (66%)
    Male/Masculine 73 (15%) 66 (13%) 72 (16%)
Number of challenges


    0 98 (20%) 99 (20%) 85 (18%)
    1 276 (57%) 297 (61%) 275 (59%)
    >=2 114 (23%) 93 (19%) 104 (22%)
Type of challenges


    No impact 98 (20%) 99 (20%) 85 (18%)
    Other 119 (24%) 97 (20%) 124 (27%)
    School 271 (56%) 293 (60%) 255 (55%)
Type of coping strategies


    Combined 58 (12%) 61 (12%) 53 (11%)
    No action 228 (47%) 235 (48%) 236 (51%)
    Positive 202 (41%) 193 (39%) 175 (38%)
1 Mean (SD); n (%)
## % latex table generated in R 4.3.1 by xtable 1.8-4 package
## % Sat Aug 10 08:51:03 2024
## \begin{table}[ht]
## \centering
## \begin{tabular}{llll}
##   \hline
## **Demographics** & **Placebo Control**  
## N = 488 & **Project ABC**  
## N = 489 & **Project Personality**  
## N = 464 \\ 
##   \hline
## Baseline CDI mean score(0-2) & 1.16 (0.35) & 1.15 (0.34) & 1.17 (0.36) \\ 
##   Race &  &  &  \\ 
##   Asian Including Asian Desi & 50 (10\%) & 58 (12\%) & 50 (11\%) \\ 
##   Black/African-American & 33 (6.8\%) & 40 (8.2\%) & 36 (7.8\%) \\ 
##   Hispanic/Latinx & 57 (12\%) & 61 (12\%) & 53 (11\%) \\ 
##   Mixed & 74 (15\%) & 68 (14\%) & 63 (14\%) \\ 
##   White & 274 (56\%) & 262 (54\%) & 262 (56\%) \\ 
##   Age (yrs) &  &  &  \\ 
##   13 & 28 (5.7\%) & 32 (6.5\%) & 28 (6.0\%) \\ 
##   14 & 77 (16\%) & 81 (17\%) & 63 (14\%) \\ 
##   15 & 150 (31\%) & 156 (32\%) & 162 (35\%) \\ 
##   16 & 233 (48\%) & 220 (45\%) & 211 (45\%) \\ 
##   Biological sex &  &  &  \\ 
##   Female & 434 (89\%) & 437 (89\%) & 418 (90\%) \\ 
##   Male & 54 (11\%) & 52 (11\%) & 46 (9.9\%) \\ 
##   Sexual orientation &  &  &  \\ 
##   Heterosexual & 108 (22\%) & 97 (20\%) & 106 (23\%) \\ 
##   LGBTQ & 309 (63\%) & 327 (67\%) & 291 (63\%) \\ 
##   Other & 71 (15\%) & 65 (13\%) & 67 (14\%) \\ 
##   Language &  &  &  \\ 
##   English & 476 (98\%) & 476 (97\%) & 450 (97\%) \\ 
##   Other & 12 (2.5\%) & 13 (2.7\%) & 14 (3.0\%) \\ 
##   Gender identity &  &  &  \\ 
##   Non-binary & 102 (21\%) & 98 (20\%) & 86 (19\%) \\ 
##   Women/girls & 313 (64\%) & 325 (66\%) & 306 (66\%) \\ 
##   Male/Masculine & 73 (15\%) & 66 (13\%) & 72 (16\%) \\ 
##   Number of challenges &  &  &  \\ 
##   0 & 98 (20\%) & 99 (20\%) & 85 (18\%) \\ 
##   1 & 276 (57\%) & 297 (61\%) & 275 (59\%) \\ 
##   $>$=2 & 114 (23\%) & 93 (19\%) & 104 (22\%) \\ 
##   Type of challenges &  &  &  \\ 
##   No impact & 98 (20\%) & 99 (20\%) & 85 (18\%) \\ 
##   Other & 119 (24\%) & 97 (20\%) & 124 (27\%) \\ 
##   School & 271 (56\%) & 293 (60\%) & 255 (55\%) \\ 
##   Type of coping strategies &  &  &  \\ 
##   Combined & 58 (12\%) & 61 (12\%) & 53 (11\%) \\ 
##   No action & 228 (47\%) & 235 (48\%) & 236 (51\%) \\ 
##   Positive & 202 (41\%) & 193 (39\%) & 175 (38\%) \\ 
##    \hline
## \end{tabular}
## \end{table}

3 Baseline CDI Prediction Model

Will use risk-based approach.

The first step is to construct the baseline CDI prediction model. The focus of the “risk” prediction model is on accurately predicting individuals’ “disease” risk. Several considerations need to be taken into account:

Statistical Model Selection:

Consideration of models: will use linear model (simple and interpretability)

Performance Metrics:

For continuous outcomes, models will be evaluated and selected based on root mean squared error (RMSE), calibration slope, and calibration-at-large.

Overfitting:

Employ a leave-one-out cross-validation (LOOCV) framework to address overfitting. LOOCV will be conducted on 80% of the samples (derivation cohort), while the remaining 20% will serve as a test cohort to mimic an external validation. Final model will be constructed on the entire dataset.

a glimpse of the variables:

## tibble [977 × 11] (S3: tbl_df/tbl/data.frame)
##  $ b_response_id : chr [1:977] "R_02uyQw7i0v8yQRX" "R_089oJuXofDsITq9" "R_08lyOxzUAzbJqcV" "R_0lDh6r7xrSRAemR" ...
##  $ condition     : Factor w/ 2 levels "Placebo Control",..: 1 1 1 2 1 1 1 2 2 2 ...
##  $ b_cdi_mean    : num [1:977] 1.417 1.167 1.333 1 0.917 ...
##  $ f1_cdi_mean   : num [1:977] 1 0.583 1.417 0.833 0.583 ...
##  $ recode_race   : Factor w/ 5 levels "Asian Including Asian Desi",..: 5 5 5 1 5 4 5 5 5 5 ...
##  $ b_screener_age: num [1:977] 14 16 15 14 16 14 14 16 16 16 ...
##  $ b_dem_sex     : Factor w/ 2 levels "Female","Male": 1 1 1 1 1 1 1 1 1 1 ...
##  $ recode_orient : Factor w/ 3 levels "Heterosexual",..: 2 2 2 1 1 2 1 1 2 2 ...
##  $ genderid3     : Factor w/ 3 levels "1","2","3": 1 2 2 2 2 2 2 2 2 2 ...
##  $ family_cat    : Factor w/ 3 levels "No impact","Other",..: 2 1 3 3 3 2 3 2 3 2 ...
##  $ cope3         : Factor w/ 3 levels "1","2","3": 3 3 2 2 2 2 2 2 2 2 ...

3.1 Build prediction model

Build prediction models using caret

Summary of baseline model output of project ABC model:

##             Length Class      Mode     
## a0           100   -none-     numeric  
## beta        1300   dgCMatrix  S4       
## df           100   -none-     numeric  
## dim            2   -none-     numeric  
## lambda       100   -none-     numeric  
## dev.ratio    100   -none-     numeric  
## nulldev        1   -none-     numeric  
## npasses        1   -none-     numeric  
## jerr           1   -none-     numeric  
## offset         1   -none-     logical  
## call           5   -none-     call     
## nobs           1   -none-     numeric  
## lambdaOpt      1   -none-     numeric  
## xNames        13   -none-     character
## problemType    1   -none-     character
## tuneValue      2   data.frame list     
## obsLevels      1   -none-     logical  
## param          0   -none-     list

Summary of baseline model output of project personality model:

##             Length Class      Mode     
## a0           65    -none-     numeric  
## beta        845    dgCMatrix  S4       
## df           65    -none-     numeric  
## dim           2    -none-     numeric  
## lambda       65    -none-     numeric  
## dev.ratio    65    -none-     numeric  
## nulldev       1    -none-     numeric  
## npasses       1    -none-     numeric  
## jerr          1    -none-     numeric  
## offset        1    -none-     logical  
## call          5    -none-     call     
## nobs          1    -none-     numeric  
## lambdaOpt     1    -none-     numeric  
## xNames       13    -none-     character
## problemType   1    -none-     character
## tuneValue     2    data.frame list     
## obsLevels     1    -none-     logical  
## param         0    -none-     list

3.2 Evaluate model performance

Summaries the model performance in the following table:

Table 3.1: Model performance of baseline risk model (project ABC)
Metric Validation Test
RMSE 0.3992314 0.3959155
Calibration Slope 1.1575945 0.9232901
Calibration at Large -0.1543406 0.0670571
Table 3.1: Model performance of baseline risk model (project personality)
Metric Validation Test
RMSE 0.4055562 0.4266310
Calibration Slope 1.0819270 0.0040360
Calibration at Large -0.0809549 0.9874272

3.3 Final baseline prediciton model

The final model is constructed on the entire dataset

## 14 x 1 sparse Matrix of class "dgCMatrix"
##                                             s0
## (Intercept)                        1.024079643
## recode_raceBlack/African-American -0.029861721
## recode_raceHispanic/Latinx        -0.067205445
## recode_raceMixed                  -0.008337803
## recode_raceWhite                   0.041071994
## b_dem_sexMale                     -0.121873161
## recode_orientLGBTQ                 0.073487928
## recode_orientOther                 0.102386860
## genderid32                        -0.109173072
## genderid33                        -0.018497319
## family_catOther                    0.060132421
## family_catSchool                   0.020780601
## cope32                            -0.024043888
## cope33                            -0.112895337
## 14 x 1 sparse Matrix of class "dgCMatrix"
##                                             s0
## (Intercept)                        1.152897654
## recode_raceBlack/African-American -0.054500296
## recode_raceHispanic/Latinx        -0.048753462
## recode_raceMixed                  -0.030923716
## recode_raceWhite                  -0.002056701
## b_dem_sexMale                     -0.104694777
## recode_orientLGBTQ                 0.068796666
## recode_orientOther                 0.064098555
## genderid32                        -0.146439033
## genderid33                        -0.102595869
## family_catOther                    0.020356363
## family_catSchool                  -0.015311446
## cope32                            -0.054863363
## cope33                            -0.125546490

3.3.1 Project ABC baseline model

A summary of the baseline prediction model, for project ABC:

  Dependent variable
Predictors Estimates CI p
(Intercept) 1.06 0.92 – 1.21 <0.001
recode race
[Black/African-American]
-0.04 -0.16 – 0.08 0.527
recode race
[Hispanic/Latinx]
-0.08 -0.19 – 0.02 0.129
recode race [Mixed] -0.03 -0.13 – 0.08 0.617
recode race [White] 0.03 -0.05 – 0.12 0.462
b dem sex [Male] -0.15 -0.25 – -0.04 0.007
recode orient [LGBTQ] 0.09 0.02 – 0.16 0.010
recode orient [Other] 0.12 0.03 – 0.22 0.008
genderid3 [2] -0.13 -0.20 – -0.06 <0.001
genderid3 [3] -0.02 -0.12 – 0.08 0.687
family cat [Other] 0.07 -0.01 – 0.15 0.070
family cat [School] 0.03 -0.03 – 0.10 0.313
cope3 [2] -0.06 -0.14 – 0.02 0.137
cope3 [3] -0.16 -0.24 – -0.08 <0.001
Observations 977
R2 / R2 adjusted 0.079 / 0.067

3.3.2 Project personality baseline model

for project personality:
  Dependent variable
Predictors Estimates CI p
(Intercept) 1.16 1.00 – 1.31 <0.001
recode race
[Black/African-American]
-0.06 -0.18 – 0.07 0.387
recode race
[Hispanic/Latinx]
-0.05 -0.16 – 0.06 0.382
recode race [Mixed] -0.03 -0.14 – 0.08 0.552
recode race [White] -0.00 -0.09 – 0.09 0.937
b dem sex [Male] -0.11 -0.22 – 0.01 0.066
recode orient [LGBTQ] 0.07 0.00 – 0.14 0.046
recode orient [Other] 0.06 -0.03 – 0.16 0.182
genderid3 [2] -0.15 -0.22 – -0.08 <0.001
genderid3 [3] -0.10 -0.21 – -0.00 0.049
family cat [Other] 0.02 -0.06 – 0.10 0.625
family cat [School] -0.02 -0.09 – 0.05 0.664
cope3 [2] -0.06 -0.15 – 0.03 0.194
cope3 [3] -0.13 -0.22 – -0.04 0.005
Observations 952
R2 / R2 adjusted 0.052 / 0.039

3.4 LRT

LRT was used to test the sig. of variables in the baseline model

For project ABC:

Variable Deviance DF P_value
Gender identity 15.456260 2 0.00
Race 9.522943 4 0.05
Biological sex 7.509756 1 0.01
Sexual Orientation 8.728432 2 0.01
Challenges 3.339847 2 0.19
Coping strategies 19.418374 2 0.00

For project personality

Variable Deviance DF P_value
Gender identity 16.358782 2 0.00
Race 2.266644 4 0.69
Biological sex 3.441522 1 0.06
Sexual Orientation 4.084130 2 0.13
Challenges 1.220614 2 0.54
Coping strategies 10.608348 2 0.00

For Project Personality

4 Investigate HTE

4.1 Main effect only model

first, replicate the model in the original paper for comparison. the main effect model which adjusted for baseline CDI score is specified as:\[E(Y|\bf{X}) = \text{baseline CDI}+ \text{condition}\]

print the model, compare with the published paper (results are similar): for project ABC vs control:

## 
## Call:
## lm(formula = f1_cdi_mean ~ b_cdi_mean + condition, data = abc_main_dat)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -1.18567 -0.21921  0.01075  0.22509  1.04660 
## 
## Coefficients:
##                      Estimate Std. Error t value Pr(>|t|)    
## (Intercept)           0.27471    0.04097   6.705 3.42e-11 ***
## b_cdi_mean            0.64303    0.03263  19.706  < 2e-16 ***
## conditionProject ABC -0.07953    0.02234  -3.560 0.000389 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.3492 on 974 degrees of freedom
## Multiple R-squared:  0.2922, Adjusted R-squared:  0.2907 
## F-statistic:   201 on 2 and 974 DF,  p-value: < 2.2e-16
  f 1 cdi mean
Predictors Estimates CI p
(Intercept) 0.27 0.19 – 0.36 <0.001
b cdi mean 0.64 0.58 – 0.71 <0.001
condition [Project ABC] -0.08 -0.12 – -0.04 <0.001
Observations 977
R2 / R2 adjusted 0.292 / 0.291

for project personality vs control:

## 
## Call:
## lm(formula = f1_cdi_mean ~ b_cdi_mean + condition, data = person_main_dat)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -1.17752 -0.22054  0.01222  0.23434  1.03109 
## 
## Coefficients:
##                              Estimate Std. Error t value Pr(>|t|)    
## (Intercept)                   0.31125    0.04188   7.432 2.38e-13 ***
## b_cdi_mean                    0.61148    0.03329  18.366  < 2e-16 ***
## conditionProject Personality -0.06895    0.02337  -2.950  0.00325 ** 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.3604 on 949 degrees of freedom
## Multiple R-squared:  0.2664, Adjusted R-squared:  0.2649 
## F-statistic: 172.3 on 2 and 949 DF,  p-value: < 2.2e-16
  f 1 cdi mean
Predictors Estimates CI p
(Intercept) 0.31 0.23 – 0.39 <0.001
b cdi mean 0.61 0.55 – 0.68 <0.001
condition [Project
Personality]
-0.07 -0.11 – -0.02 0.003
Observations 952
R2 / R2 adjusted 0.266 / 0.265

4.2 HTE model

The HTE is defined as: \[E(Y|\bf{X}) = \text{baseline CDI}+ \text{condition} + \text{lp}+\text{condition}\times \text{lp}\] where lp is the linear predictor of the baseline model.

4.2.1 HTE Project ABC

The hte model for Project ABC vs control:

## 
## Call:
## lm(formula = f1_cdi_mean ~ b_cdi_mean + condition * lp_abc, data = dat_abc)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -1.16587 -0.21245  0.01587  0.22319  1.04761 
## 
## Coefficients:
##                             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)                  0.20672    0.13296   1.555   0.1203    
## b_cdi_mean                   0.60042    0.03517  17.070   <2e-16 ***
## conditionProject ABC        -0.46326    0.18742  -2.472   0.0136 *  
## lp_abc                       0.11985    0.14022   0.855   0.3929    
## conditionProject ABC:lp_abc  0.39263    0.19023   2.064   0.0393 *  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.3471 on 972 degrees of freedom
## Multiple R-squared:  0.302,  Adjusted R-squared:  0.2991 
## F-statistic: 105.1 on 4 and 972 DF,  p-value: < 2.2e-16
  f 1 cdi mean
Predictors Estimates CI p
(Intercept) 0.21 -0.05 – 0.47 0.120
b cdi mean 0.60 0.53 – 0.67 <0.001
condition [Project ABC] -0.46 -0.83 – -0.10 0.014
lp abc 0.12 -0.16 – 0.40 0.393
condition [Project ABC] ×
lp abc
0.39 0.02 – 0.77 0.039
Observations 977
R2 / R2 adjusted 0.302 / 0.299

4.2.2 HTE Project personality

The hte model for Project personality vs control:

## 
## Call:
## lm(formula = f1_cdi_mean ~ b_cdi_mean + condition * lp_person, 
##     data = dat_person)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -1.1604 -0.2189  0.0176  0.2321  1.0427 
## 
## Coefficients:
##                                        Estimate Std. Error t value Pr(>|t|)    
## (Intercept)                             0.18881    0.16684   1.132    0.258    
## b_cdi_mean                              0.58760    0.03544  16.580   <2e-16 ***
## conditionProject Personality           -0.28385    0.24270  -1.170    0.242    
## lp_person                               0.15184    0.17399   0.873    0.383    
## conditionProject Personality:lp_person  0.21744    0.24431   0.890    0.374    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.3599 on 947 degrees of freedom
## Multiple R-squared:   0.27,  Adjusted R-squared:  0.2669 
## F-statistic: 87.56 on 4 and 947 DF,  p-value: < 2.2e-16
  f 1 cdi mean
Predictors Estimates CI p
(Intercept) 0.19 -0.14 – 0.52 0.258
b cdi mean 0.59 0.52 – 0.66 <0.001
condition [Project
Personality]
-0.28 -0.76 – 0.19 0.242
lp person 0.15 -0.19 – 0.49 0.383
condition [Project
Personality] × lp person
0.22 -0.26 – 0.70 0.374
Observations 952
R2 / R2 adjusted 0.270 / 0.267

5 Evaluate HTE

The HTE is evaluated using calibration plot.

5.1 Point estiamtes

First, report the point estiamtes of ATE and cATE.

(#tab:abc_cali)summary of avearged treatment effect
Comparsion ATE SE
conditionProject ABC Project ABC vs Control -0.0795347 0.0223413
conditionProject Personality Project personality vs Control -0.0689513 0.0233696

A brief summary of the linear predictors:

## Length  Class   Mode 
##      0   NULL   NULL
##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##  0.7486  0.9316  0.9751  0.9888  1.0464  1.2441

5.2 Bootstrapped CI

compute bootstrapped CI for cATE:

for Project personality:

5.3 Calibration plots

The linear predictor entered model as a continuous variable. For presentation purpose, the linear predictor is discretized into five “risk” groups using quantiles (0.2 incremental). The averaged HTEs/cATEs by “risk” group are calculated and compared with the ATE.

For Project ABC vs control:

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##  0.6620  0.8980  0.9743  0.9782  1.0605  1.2931

For Project personality vs control:

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##  0.7486  0.9316  0.9751  0.9888  1.0464  1.2441

combined plots for reporting purpose:

5.3.1 Rearrange for report (For Porject ABC vs control)

Rearrange tables for reporting purpose:

## Joining with `by = join_by(quantile_grp)`

The calibration table and plot:

5.3.2 Rearrange for report (For Porject personality vs control)

## Joining with `by = join_by(quantile_grp)`

The calibration table and plot:

5.4 Distribution in high risk groups

## % latex table generated in R 4.3.1 by xtable 1.8-4 package
## % Sat Aug 10 08:52:59 2024
## \begin{table}[ht]
## \centering
## \begin{tabular}{lllllllllll}
##   \hline
## **Demographics** & **1**  
## N = 98 & **10**  
## N = 94 & **2**  
## N = 104 & **3**  
## N = 92 & **4**  
## N = 97 & **5**  
## N = 101 & **6**  
## N = 95 & **7**  
## N = 103 & **8**  
## N = 91 & **9**  
## N = 102 \\ 
##   \hline
## condition &  &  &  &  &  &  &  &  &  &  \\ 
##   Placebo Control & 44 (45\%) & 42 (45\%) & 57 (55\%) & 52 (57\%) & 51 (53\%) & 44 (44\%) & 41 (43\%) & 51 (50\%) & 45 (49\%) & 61 (60\%) \\ 
##   Project ABC & 54 (55\%) & 52 (55\%) & 47 (45\%) & 40 (43\%) & 46 (47\%) & 57 (56\%) & 54 (57\%) & 52 (50\%) & 46 (51\%) & 41 (40\%) \\ 
##   Baseline CDI mean score(0-2) & 0.92 (0.31) & 1.37 (0.30) & 1.04 (0.37) & 1.06 (0.33) & 1.05 (0.29) & 1.14 (0.34) & 1.23 (0.31) & 1.21 (0.30) & 1.26 (0.32) & 1.29 (0.30) \\ 
##   Race &  &  &  &  &  &  &  &  &  &  \\ 
##   Asian Including Asian Desi & 18 (18\%) & 1 (1.1\%) & 13 (13\%) & 35 (38\%) & 3 (3.1\%) & 9 (8.9\%) & 10 (11\%) & 9 (8.7\%) & 5 (5.5\%) & 5 (4.9\%) \\ 
##   Black/African-American & 16 (16\%) & 4 (4.3\%) & 16 (15\%) & 5 (5.4\%) & 5 (5.2\%) & 11 (11\%) & 8 (8.4\%) & 3 (2.9\%) & 3 (3.3\%) & 2 (2.0\%) \\ 
##   Hispanic/Latinx & 38 (39\%) & 1 (1.1\%) & 27 (26\%) & 1 (1.1\%) & 16 (16\%) & 20 (20\%) & 2 (2.1\%) & 3 (2.9\%) & 5 (5.5\%) & 5 (4.9\%) \\ 
##   Mixed & 10 (10\%) & 14 (15\%) & 26 (25\%) & 1 (1.1\%) & 13 (13\%) & 17 (17\%) & 21 (22\%) & 11 (11\%) & 12 (13\%) & 17 (17\%) \\ 
##   White & 16 (16\%) & 74 (79\%) & 22 (21\%) & 50 (54\%) & 60 (62\%) & 44 (44\%) & 54 (57\%) & 77 (75\%) & 66 (73\%) & 73 (72\%) \\ 
##   Age (yrs) &  &  &  &  &  &  &  &  &  &  \\ 
##   13 & 3 (3.1\%) & 10 (11\%) & 4 (3.8\%) & 1 (1.1\%) & 5 (5.2\%) & 6 (5.9\%) & 7 (7.4\%) & 8 (7.8\%) & 9 (9.9\%) & 7 (6.9\%) \\ 
##   14 & 12 (12\%) & 16 (17\%) & 13 (13\%) & 19 (21\%) & 11 (11\%) & 21 (21\%) & 15 (16\%) & 12 (12\%) & 20 (22\%) & 19 (19\%) \\ 
##   15 & 32 (33\%) & 30 (32\%) & 30 (29\%) & 29 (32\%) & 27 (28\%) & 28 (28\%) & 27 (28\%) & 39 (38\%) & 27 (30\%) & 37 (36\%) \\ 
##   16 & 51 (52\%) & 38 (40\%) & 57 (55\%) & 43 (47\%) & 54 (56\%) & 46 (46\%) & 46 (48\%) & 44 (43\%) & 35 (38\%) & 39 (38\%) \\ 
##   Biological sex &  &  &  &  &  &  &  &  &  &  \\ 
##   Female & 66 (67\%) & 94 (100\%) & 83 (80\%) & 80 (87\%) & 87 (90\%) & 92 (91\%) & 80 (84\%) & 101 (98\%) & 87 (96\%) & 101 (99\%) \\ 
##   Male & 32 (33\%) & 0 (0\%) & 21 (20\%) & 12 (13\%) & 10 (10\%) & 9 (8.9\%) & 15 (16\%) & 2 (1.9\%) & 4 (4.4\%) & 1 (1.0\%) \\ 
##   Sexual orientation &  &  &  &  &  &  &  &  &  &  \\ 
##   Heterosexual & 77 (79\%) & 0 (0\%) & 45 (43\%) & 39 (42\%) & 5 (5.2\%) & 20 (20\%) & 16 (17\%) & 0 (0\%) & 3 (3.3\%) & 0 (0\%) \\ 
##   LGBTQ & 20 (20\%) & 64 (68\%) & 57 (55\%) & 48 (52\%) & 83 (86\%) & 68 (67\%) & 75 (79\%) & 90 (87\%) & 54 (59\%) & 77 (75\%) \\ 
##   Other & 1 (1.0\%) & 30 (32\%) & 2 (1.9\%) & 5 (5.4\%) & 9 (9.3\%) & 13 (13\%) & 4 (4.2\%) & 13 (13\%) & 34 (37\%) & 25 (25\%) \\ 
##   Language &  &  &  &  &  &  &  &  &  &  \\ 
##   English & 91 (93\%) & 93 (99\%) & 100 (96\%) & 88 (96\%) & 94 (97\%) & 99 (98\%) & 94 (99\%) & 102 (99\%) & 91 (100\%) & 100 (98\%) \\ 
##   Other & 7 (7.1\%) & 1 (1.1\%) & 4 (3.8\%) & 4 (4.3\%) & 3 (3.1\%) & 2 (2.0\%) & 1 (1.1\%) & 1 (1.0\%) & 0 (0\%) & 2 (2.0\%) \\ 
##   Gender identity &  &  &  &  &  &  &  &  &  &  \\ 
##   Non-binary & 1 (1.0\%) & 79 (84\%) & 2 (1.9\%) & 1 (1.1\%) & 5 (5.2\%) & 5 (5.0\%) & 20 (21\%) & 10 (9.7\%) & 27 (30\%) & 50 (49\%) \\ 
##   Women/girls & 70 (71\%) & 0 (0\%) & 83 (80\%) & 80 (87\%) & 86 (89\%) & 88 (87\%) & 61 (64\%) & 87 (84\%) & 48 (53\%) & 35 (34\%) \\ 
##   Male/Masculine & 27 (28\%) & 15 (16\%) & 19 (18\%) & 11 (12\%) & 6 (6.2\%) & 8 (7.9\%) & 14 (15\%) & 6 (5.8\%) & 16 (18\%) & 17 (17\%) \\ 
##   Number of challenges &  &  &  &  &  &  &  &  &  &  \\ 
##   0 & 26 (27\%) & 8 (8.5\%) & 31 (30\%) & 37 (40\%) & 9 (9.3\%) & 9 (8.9\%) & 30 (32\%) & 16 (16\%) & 17 (19\%) & 14 (14\%) \\ 
##   1 & 58 (59\%) & 44 (47\%) & 65 (63\%) & 46 (50\%) & 77 (79\%) & 58 (57\%) & 44 (46\%) & 74 (72\%) & 47 (52\%) & 60 (59\%) \\ 
##   $>$=2 & 14 (14\%) & 42 (45\%) & 8 (7.7\%) & 9 (9.8\%) & 11 (11\%) & 34 (34\%) & 21 (22\%) & 13 (13\%) & 27 (30\%) & 28 (27\%) \\ 
##   Type of challenges &  &  &  &  &  &  &  &  &  &  \\ 
##   No impact & 26 (27\%) & 8 (8.5\%) & 31 (30\%) & 37 (40\%) & 9 (9.3\%) & 9 (8.9\%) & 30 (32\%) & 16 (16\%) & 17 (19\%) & 14 (14\%) \\ 
##   Other & 11 (11\%) & 45 (48\%) & 10 (9.6\%) & 12 (13\%) & 9 (9.3\%) & 33 (33\%) & 27 (28\%) & 6 (5.8\%) & 25 (27\%) & 38 (37\%) \\ 
##   School & 61 (62\%) & 41 (44\%) & 63 (61\%) & 43 (47\%) & 79 (81\%) & 59 (58\%) & 38 (40\%) & 81 (79\%) & 49 (54\%) & 50 (49\%) \\ 
##   Type of coping strategies &  &  &  &  &  &  &  &  &  &  \\ 
##   Combined & 0 (0\%) & 35 (37\%) & 0 (0\%) & 0 (0\%) & 2 (2.1\%) & 7 (6.9\%) & 9 (9.5\%) & 13 (13\%) & 17 (19\%) & 36 (35\%) \\ 
##   No action & 11 (11\%) & 59 (63\%) & 39 (38\%) & 33 (36\%) & 25 (26\%) & 62 (61\%) & 67 (71\%) & 78 (76\%) & 47 (52\%) & 42 (41\%) \\ 
##   Positive & 87 (89\%) & 0 (0\%) & 65 (63\%) & 59 (64\%) & 70 (72\%) & 32 (32\%) & 19 (20\%) & 12 (12\%) & 27 (30\%) & 24 (24\%) \\ 
##   abc\_ate &  &  &  &  &  &  &  &  &  &  \\ 
##   -0.0795346966023499 & 98 (100\%) & 94 (100\%) & 104 (100\%) & 92 (100\%) & 97 (100\%) & 101 (100\%) & 95 (100\%) & 103 (100\%) & 91 (100\%) & 102 (100\%) \\ 
##   abc\_hte & -0.16 (0.01) & 0.00 (0.02) & -0.13 (0.01) & -0.11 (0.00) & -0.10 (0.00) & -0.09 (0.00) & -0.07 (0.00) & -0.06 (0.00) & -0.05 (0.00) & -0.03 (0.01) \\ 
##   lp\_abc & 0.78 (0.03) & 1.19 (0.04) & 0.86 (0.02) & 0.90 (0.01) & 0.92 (0.01) & 0.96 (0.01) & 1.00 (0.01) & 1.03 (0.00) & 1.06 (0.01) & 1.11 (0.02) \\ 
##    \hline
## \end{tabular}
## \end{table}
## % latex table generated in R 4.3.1 by xtable 1.8-4 package
## % Sat Aug 10 08:53:02 2024
## \begin{table}[ht]
## \centering
## \begin{tabular}{lllllllllll}
##   \hline
## **Demographics** & **1**  
## N = 101 & **10**  
## N = 96 & **2**  
## N = 94 & **3**  
## N = 103 & **4**  
## N = 83 & **5**  
## N = 95 & **6**  
## N = 120 & **7**  
## N = 72 & **8**  
## N = 94 & **9**  
## N = 94 \\ 
##   \hline
## condition &  &  &  &  &  &  &  &  &  &  \\ 
##   Placebo Control & 56 (55\%) & 46 (48\%) & 53 (56\%) & 46 (45\%) & 44 (53\%) & 47 (49\%) & 63 (53\%) & 33 (46\%) & 46 (49\%) & 54 (57\%) \\ 
##   Project Personality & 45 (45\%) & 50 (52\%) & 41 (44\%) & 57 (55\%) & 39 (47\%) & 48 (51\%) & 57 (48\%) & 39 (54\%) & 48 (51\%) & 40 (43\%) \\ 
##   Baseline CDI mean score(0-2) & 0.93 (0.32) & 1.35 (0.30) & 1.02 (0.39) & 1.06 (0.33) & 1.12 (0.37) & 1.19 (0.31) & 1.19 (0.28) & 1.25 (0.30) & 1.24 (0.36) & 1.29 (0.33) \\ 
##   Race &  &  &  &  &  &  &  &  &  &  \\ 
##   Asian Including Asian Desi & 11 (11\%) & 6 (6.3\%) & 9 (9.6\%) & 3 (2.9\%) & 26 (31\%) & 15 (16\%) & 5 (4.2\%) & 15 (21\%) & 2 (2.1\%) & 8 (8.5\%) \\ 
##   Black/African-American & 14 (14\%) & 3 (3.1\%) & 12 (13\%) & 12 (12\%) & 8 (9.6\%) & 5 (5.3\%) & 4 (3.3\%) & 3 (4.2\%) & 2 (2.1\%) & 6 (6.4\%) \\ 
##   Hispanic/Latinx & 20 (20\%) & 3 (3.1\%) & 26 (28\%) & 8 (7.8\%) & 0 (0\%) & 24 (25\%) & 13 (11\%) & 5 (6.9\%) & 3 (3.2\%) & 8 (8.5\%) \\ 
##   Mixed & 9 (8.9\%) & 14 (15\%) & 23 (24\%) & 10 (9.7\%) & 9 (11\%) & 12 (13\%) & 9 (7.5\%) & 19 (26\%) & 17 (18\%) & 15 (16\%) \\ 
##   White & 47 (47\%) & 70 (73\%) & 24 (26\%) & 70 (68\%) & 40 (48\%) & 39 (41\%) & 89 (74\%) & 30 (42\%) & 70 (74\%) & 57 (61\%) \\ 
##   Age (yrs) &  &  &  &  &  &  &  &  &  &  \\ 
##   13 & 3 (3.0\%) & 7 (7.3\%) & 4 (4.3\%) & 1 (1.0\%) & 3 (3.6\%) & 10 (11\%) & 9 (7.5\%) & 7 (9.7\%) & 7 (7.4\%) & 5 (5.3\%) \\ 
##   14 & 15 (15\%) & 17 (18\%) & 11 (12\%) & 18 (17\%) & 15 (18\%) & 9 (9.5\%) & 15 (13\%) & 13 (18\%) & 13 (14\%) & 14 (15\%) \\ 
##   15 & 39 (39\%) & 28 (29\%) & 28 (30\%) & 34 (33\%) & 25 (30\%) & 31 (33\%) & 37 (31\%) & 23 (32\%) & 32 (34\%) & 35 (37\%) \\ 
##   16 & 44 (44\%) & 44 (46\%) & 51 (54\%) & 50 (49\%) & 40 (48\%) & 45 (47\%) & 59 (49\%) & 29 (40\%) & 42 (45\%) & 40 (43\%) \\ 
##   Biological sex &  &  &  &  &  &  &  &  &  &  \\ 
##   Female & 58 (57\%) & 96 (100\%) & 77 (82\%) & 94 (91\%) & 78 (94\%) & 87 (92\%) & 113 (94\%) & 71 (99\%) & 87 (93\%) & 91 (97\%) \\ 
##   Male & 43 (43\%) & 0 (0\%) & 17 (18\%) & 9 (8.7\%) & 5 (6.0\%) & 8 (8.4\%) & 7 (5.8\%) & 1 (1.4\%) & 7 (7.4\%) & 3 (3.2\%) \\ 
##   Sexual orientation &  &  &  &  &  &  &  &  &  &  \\ 
##   Heterosexual & 86 (85\%) & 0 (0\%) & 44 (47\%) & 29 (28\%) & 27 (33\%) & 18 (19\%) & 7 (5.8\%) & 3 (4.2\%) & 0 (0\%) & 0 (0\%) \\ 
##   LGBTQ & 13 (13\%) & 66 (69\%) & 45 (48\%) & 66 (64\%) & 51 (61\%) & 63 (66\%) & 87 (73\%) & 58 (81\%) & 76 (81\%) & 75 (80\%) \\ 
##   Other & 2 (2.0\%) & 30 (31\%) & 5 (5.3\%) & 8 (7.8\%) & 5 (6.0\%) & 14 (15\%) & 26 (22\%) & 11 (15\%) & 18 (19\%) & 19 (20\%) \\ 
##   Language &  &  &  &  &  &  &  &  &  &  \\ 
##   English & 94 (93\%) & 94 (98\%) & 90 (96\%) & 100 (97\%) & 80 (96\%) & 92 (97\%) & 119 (99\%) & 70 (97\%) & 94 (100\%) & 93 (99\%) \\ 
##   Other & 7 (6.9\%) & 2 (2.1\%) & 4 (4.3\%) & 3 (2.9\%) & 3 (3.6\%) & 3 (3.2\%) & 1 (0.8\%) & 2 (2.8\%) & 0 (0\%) & 1 (1.1\%) \\ 
##   Gender identity &  &  &  &  &  &  &  &  &  &  \\ 
##   Non-binary & 0 (0\%) & 91 (95\%) & 0 (0\%) & 2 (1.9\%) & 0 (0\%) & 2 (2.1\%) & 3 (2.5\%) & 3 (4.2\%) & 20 (21\%) & 67 (71\%) \\ 
##   Women/girls & 61 (60\%) & 0 (0\%) & 77 (82\%) & 93 (90\%) & 78 (94\%) & 82 (86\%) & 95 (79\%) & 61 (85\%) & 55 (59\%) & 17 (18\%) \\ 
##   Male/Masculine & 40 (40\%) & 5 (5.2\%) & 17 (18\%) & 8 (7.8\%) & 5 (6.0\%) & 11 (12\%) & 22 (18\%) & 8 (11\%) & 19 (20\%) & 10 (11\%) \\ 
##   Number of challenges &  &  &  &  &  &  &  &  &  &  \\ 
##   0 & 10 (9.9\%) & 20 (21\%) & 27 (29\%) & 4 (3.9\%) & 37 (45\%) & 20 (21\%) & 15 (13\%) & 25 (35\%) & 11 (12\%) & 14 (15\%) \\ 
##   1 & 78 (77\%) & 39 (41\%) & 56 (60\%) & 77 (75\%) & 37 (45\%) & 48 (51\%) & 85 (71\%) & 30 (42\%) & 58 (62\%) & 43 (46\%) \\ 
##   $>$=2 & 13 (13\%) & 37 (39\%) & 11 (12\%) & 22 (21\%) & 9 (11\%) & 27 (28\%) & 20 (17\%) & 17 (24\%) & 25 (27\%) & 37 (39\%) \\ 
##   Type of challenges &  &  &  &  &  &  &  &  &  &  \\ 
##   No impact & 10 (9.9\%) & 20 (21\%) & 27 (29\%) & 4 (3.9\%) & 37 (45\%) & 20 (21\%) & 15 (13\%) & 25 (35\%) & 11 (12\%) & 14 (15\%) \\ 
##   Other & 11 (11\%) & 37 (39\%) & 10 (11\%) & 21 (20\%) & 9 (11\%) & 37 (39\%) & 19 (16\%) & 19 (26\%) & 39 (41\%) & 41 (44\%) \\ 
##   School & 80 (79\%) & 39 (41\%) & 57 (61\%) & 78 (76\%) & 37 (45\%) & 38 (40\%) & 86 (72\%) & 28 (39\%) & 44 (47\%) & 39 (41\%) \\ 
##   Type of coping strategies &  &  &  &  &  &  &  &  &  &  \\ 
##   Combined & 0 (0\%) & 31 (32\%) & 0 (0\%) & 0 (0\%) & 2 (2.4\%) & 1 (1.1\%) & 8 (6.7\%) & 15 (21\%) & 29 (31\%) & 25 (27\%) \\ 
##   No action & 16 (16\%) & 65 (68\%) & 28 (30\%) & 33 (32\%) & 36 (43\%) & 63 (66\%) & 95 (79\%) & 49 (68\%) & 50 (53\%) & 29 (31\%) \\ 
##   Positive & 85 (84\%) & 0 (0\%) & 66 (70\%) & 70 (68\%) & 45 (54\%) & 31 (33\%) & 17 (14\%) & 8 (11\%) & 15 (16\%) & 40 (43\%) \\ 
##   person\_ate &  &  &  &  &  &  &  &  &  &  \\ 
##   -0.0689513014845681 & 101 (100\%) & 96 (100\%) & 94 (100\%) & 103 (100\%) & 83 (100\%) & 95 (100\%) & 120 (100\%) & 72 (100\%) & 94 (100\%) & 94 (100\%) \\ 
##   person\_hte & -0.10 (0.01) & -0.03 (0.01) & -0.09 (0.00) & -0.08 (0.00) & -0.08 (0.00) & -0.07 (0.00) & -0.07 (0.00) & -0.06 (0.00) & -0.06 (0.00) & -0.04 (0.00) \\ 
##   lp\_person & 0.84 (0.03) & 1.17 (0.03) & 0.89 (0.01) & 0.93 (0.01) & 0.94 (0.01) & 0.96 (0.01) & 1.00 (0.01) & 1.01 (0.01) & 1.05 (0.01) & 1.10 (0.02) \\ 
##    \hline
## \end{tabular}
## \end{table}

6 Sensitivity Analysis

6.1 inverse probability of missingness weighting (IPMW)

Check the missingness, and regress the missing status on variables with complete info.

## .
##    0    1 
## 1441   60
## tibble [1,501 × 10] (S3: tbl_df/tbl/data.frame)
##  $ condition    : chr [1:1501] "Placebo Control" "Project Personality" "Placebo Control" "Placebo Control" ...
##  $ b_cdi_mean   : num [1:1501] 1.417 0.917 1.167 1.333 0.917 ...
##  $ f1_cdi_mean  : num [1:1501] 1 0.583 0.583 1.417 0.5 ...
##  $ recode_race  : Factor w/ 5 levels "Asian Including Asian Desi",..: 5 5 5 5 5 5 1 1 5 4 ...
##  $ b_dem_sex    : Factor w/ 2 levels "Female","Male": 1 1 1 1 1 1 1 1 1 1 ...
##  $ recode_orient: Factor w/ 3 levels "Heterosexual",..: 2 3 2 2 2 2 1 2 1 2 ...
##  $ genderid3    : Factor w/ 3 levels "1","2","3": 1 1 2 2 2 3 2 2 2 2 ...
##  $ family_cat   : Factor w/ 3 levels "No impact","Other",..: 2 2 1 3 3 3 3 3 3 2 ...
##  $ cope3        : Factor w/ 3 levels "1","2","3": 3 3 3 2 1 2 2 2 2 2 ...
##  $ missing      : num [1:1501] 0 0 0 0 0 0 0 0 0 0 ...

A summary of the IPCW model:

## 
## Call:
## glm(formula = missing ~ genderid3 + b_cdi_mean + family_cat + 
##     cope3, family = "binomial", data = model_dat)
## 
## Coefficients:
##                  Estimate Std. Error z value Pr(>|z|)    
## (Intercept)       -3.0919     0.8430  -3.668 0.000245 ***
## genderid32        -1.2368     0.2870  -4.309 1.64e-05 ***
## genderid33        -1.2284     0.4630  -2.653 0.007978 ** 
## b_cdi_mean         0.2561     0.4193   0.611 0.541408    
## family_catOther   -0.1730     0.3659  -0.473 0.636328    
## family_catSchool  -0.5263     0.3270  -1.610 0.107490    
## cope32             0.9205     0.5452   1.688 0.091346 .  
## cope33             0.7123     0.5729   1.243 0.213729    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 503.91  on 1500  degrees of freedom
## Residual deviance: 476.46  on 1493  degrees of freedom
## AIC: 492.46
## 
## Number of Fisher Scoring iterations: 6

Summary of IPCW weights:

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##  0.9687  0.9815  0.9878  1.0000  1.0002  1.1427

6.2 Baseline prediction models

baseline model for Project ABC:
  f 1 cdi mean
Predictors Estimates CI p
(Intercept) 1.06 0.92 – 1.21 <0.001
recode race
[Black/African-American]
-0.04 -0.16 – 0.08 0.527
recode race
[Hispanic/Latinx]
-0.08 -0.19 – 0.02 0.128
recode race [Mixed] -0.03 -0.13 – 0.08 0.617
recode race [White] 0.03 -0.05 – 0.12 0.462
b dem sex [Male] -0.15 -0.25 – -0.04 0.006
recode orient [LGBTQ] 0.09 0.02 – 0.16 0.010
recode orient [Other] 0.12 0.03 – 0.22 0.008
genderid3 [2] -0.13 -0.20 – -0.06 <0.001
genderid3 [3] -0.02 -0.12 – 0.08 0.687
family cat [Other] 0.07 -0.01 – 0.15 0.070
family cat [School] 0.03 -0.03 – 0.10 0.313
cope3 [2] -0.06 -0.14 – 0.02 0.137
cope3 [3] -0.16 -0.24 – -0.08 <0.001
Observations 977
R2 0.079
baseline model for Project personality:
  f 1 cdi mean
Predictors Estimates CI p
(Intercept) 1.16 1.00 – 1.31 <0.001
recode race
[Black/African-American]
-0.06 -0.18 – 0.07 0.387
recode race
[Hispanic/Latinx]
-0.05 -0.16 – 0.06 0.382
recode race [Mixed] -0.03 -0.14 – 0.08 0.552
recode race [White] -0.00 -0.09 – 0.09 0.937
b dem sex [Male] -0.11 -0.22 – 0.01 0.065
recode orient [LGBTQ] 0.07 0.00 – 0.14 0.046
recode orient [Other] 0.06 -0.03 – 0.16 0.182
genderid3 [2] -0.15 -0.22 – -0.08 <0.001
genderid3 [3] -0.10 -0.21 – -0.00 0.049
family cat [Other] 0.02 -0.06 – 0.10 0.625
family cat [School] -0.02 -0.09 – 0.05 0.664
cope3 [2] -0.06 -0.15 – 0.03 0.193
cope3 [3] -0.13 -0.22 – -0.04 0.005
Observations 952
R2 0.052

6.3 Main effect model

Main effect model for Project ABC:
  f 1 cdi mean
Predictors Estimates CI p
(Intercept) 0.27 0.19 – 0.36 <0.001
b cdi mean 0.64 0.58 – 0.71 <0.001
condition [Project ABC] -0.08 -0.12 – -0.04 <0.001
Observations 977
R2 0.292
Main effect model for Project Personality:
  f 1 cdi mean
Predictors Estimates CI p
(Intercept) 0.31 0.23 – 0.39 <0.001
b cdi mean 0.61 0.55 – 0.68 <0.001
condition [Project
Personality]
-0.07 -0.11 – -0.02 0.003
Observations 952
R2 0.266

6.4 The HTE models

HTE model for Project ABC:
  f 1 cdi mean
Predictors Estimates CI p
(Intercept) 0.21 -0.05 – 0.47 0.120
b cdi mean 0.60 0.53 – 0.67 <0.001
condition [Project ABC] -0.46 -0.83 – -0.10 0.013
abc ipcw lp 0.12 -0.15 – 0.39 0.393
condition [Project ABC] ×
abc ipcw lp
0.39 0.02 – 0.77 0.039
Observations 977
R2 0.302
HTE model for Project Personality:
  f 1 cdi mean
Predictors Estimates CI p
(Intercept) 0.19 -0.14 – 0.52 0.258
b cdi mean 0.59 0.52 – 0.66 <0.001
condition [Project
Personality]
-0.28 -0.76 – 0.19 0.242
person ipcw lp 0.15 -0.19 – 0.49 0.383
condition [Project
Personality] × person
ipcw lp
0.22 -0.26 – 0.70 0.373
Observations 952
R2 0.270
Table 6.1: summary of avearged treatment effect
Comparsion ATE SE
conditionProject ABC Project ABC vs Control -0.0795347 0.0223413
conditionProject Personality Project personality vs Control -0.0689513 0.0233696

6.5 Risk stratification

6.6 Boot CI

For Project ABC:

For Project personality:

6.7 Risk stratification

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##  0.6620  0.8980  0.9743  0.9782  1.0605  1.2931

Distribution of baseline depression severity score

7 Appendix

The original data summary statistics:

Demographics Treatment Received
Placebo Control
N = 505
1
Project ABC
N = 514
1
Project Personality
N = 482
1
recode_race


    Asian Including Asian Desi 50 (9.9%) 59 (11%) 50 (10%)
    Black/African-American 33 (6.5%) 41 (8.0%) 36 (7.5%)
    Hispanic/Latinx 57 (11%) 62 (12%) 55 (11%)
    Mixed 76 (15%) 70 (14%) 63 (13%)
    Prefer not to answer 12 (2.4%) 9 (1.8%) 12 (2.5%)
    White 277 (55%) 273 (53%) 266 (55%)
Agender 11 (2.2%) 7 (1.4%) 11 (2.3%)
Not sure/Questioning 39 (7.7%) 31 (6.0%) 32 (6.6%)
Unspecified Gender 15 (3.0%) 13 (2.5%) 12 (2.5%)
Androgynous 27 (5.3%) 28 (5.4%) 31 (6.4%)
Non-binary 68 (13%) 71 (14%) 68 (14%)
Two-spirited 0 (0%) 4 (0.8%) 4 (0.8%)
Transgender - Female to Male 36 (7.1%) 35 (6.8%) 38 (7.9%)
Trans Female/Trans Feminine 6 (1.2%) 8 (1.6%) 3 (0.6%)
Trans Male/Trans Masculine 40 (7.9%) 38 (7.4%) 38 (7.9%)
Gender Expansive 3 (0.6%) 6 (1.2%) 4 (0.8%)
Third Gender 1 (0.2%) 1 (0.2%) 2 (0.4%)
Genderqueer 33 (6.5%) 22 (4.3%) 23 (4.8%)
Transgender - Male to Female 4 (0.8%) 3 (0.6%) 3 (0.6%)
Man/Boy 75 (15%) 69 (13%) 77 (16%)
Transgender 45 (8.9%) 33 (6.4%) 38 (7.9%)
Woman/Girl 329 (65%) 337 (66%) 323 (67%)
b_screener_age


    13 28 (5.5%) 36 (7.0%) 29 (6.0%)
    14 77 (15%) 85 (17%) 65 (13%)
    15 158 (31%) 164 (32%) 170 (35%)
    16 242 (48%) 229 (45%) 218 (45%)
Biological Sex


    Female 445 (88%) 449 (87%) 429 (89%)
    Male 55 (11%) 54 (11%) 47 (9.8%)
    Other 2 (0.4%) 8 (1.6%) 2 (0.4%)
    Prefer not to say 3 (0.6%) 3 (0.6%) 4 (0.8%)
Sexual Orientation


    Asexual 24 (4.8%) 29 (5.6%) 28 (5.8%)
    Bisexual 143 (28%) 144 (28%) 123 (26%)
    Gay/Lesbian/Homosexual 55 (11%) 52 (10%) 53 (11%)
    Heterosexual/Straight 110 (22%) 101 (20%) 107 (22%)
    I do not use a label 33 (6.5%) 29 (5.6%) 24 (5.0%)
    I do not want to respond 0 (0%) 5 (1.0%) 1 (0.2%)
    Other/Not listed (please specify) 18 (3.6%) 12 (2.3%) 20 (4.1%)
    Pansexual 49 (9.7%) 58 (11%) 41 (8.5%)
    Queer 34 (6.7%) 28 (5.4%) 28 (5.8%)
    Unsure/Questioning 39 (7.7%) 56 (11%) 57 (12%)
b_covid_family_family_did_not_enough_enough_money_for_food 51 (10%) 43 (8.4%) 55 (11%)
b_covid_family_family_did_not_have_a_regular_place_to_sleep_or_stay 7 (1.4%) 5 (1.0%) 6 (1.2%)
b_covid_family_i_could_not_attend_school_in_person 358 (71%) 367 (71%) 330 (68%)
b_covid_family_i_could_not_attend_school_at_all 45 (8.9%) 32 (6.2%) 36 (7.5%)
b_covid_family_other 56 (11%) 39 (7.6%) 56 (12%)
b_covid_family_family_did_not_have_enough_money_for_gas_transportation 36 (7.1%) 20 (3.9%) 32 (6.6%)
b_covid_family_family_did_not_have_enough_money_to_pay_rent 43 (8.5%) 36 (7.0%) 41 (8.5%)
b_covid_family_the_covid_19_pandemic_has_not_affected_me_or_my_family_in_these_ways_in_the_past_2_weeks 100 (20%) 107 (21%) 91 (19%)
b_covid_cope_1_connecting_with_others 233 (46%) 227 (44%) 205 (43%)
b_covid_cope_1_including_talking_with_people_you_trust_about_your_concerns_and_how_you_are_feeling 233 (46%) 227 (44%) 205 (43%)
b_covid_cope_1_contacting_a_healthcare_provider 58 (11%) 51 (9.9%) 49 (10%)
b_covid_cope_1_drinking_alcohol 37 (7.3%) 42 (8.2%) 33 (6.8%)
b_covid_cope_1_smoking_more_cigarettes_or_vaping_more 35 (6.9%) 37 (7.2%) 33 (6.8%)
recode_language


    English 493 (98%) 499 (97%) 468 (97%)
    Other 12 (2.4%) 15 (2.9%) 14 (2.9%)
b_cdi_mean 1.16 (0.35) 1.16 (0.34) 1.17 (0.35)
recode_orient


    Heterosexual 110 (22%) 101 (20%) 107 (22%)
    I do not want to respond 0 (0%) 5 (1.0%) 1 (0.2%)
    LGBTQ 320 (63%) 338 (66%) 302 (63%)
    Other 75 (15%) 70 (14%) 72 (15%)
1 n (%); Mean (SD)